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DETAILED ACTION 



1 . This communication is responsive to an amendment filed 1 2/22/04. 

2. Claims 1-56 and 59-65 are pending in this application. Claims 1, 5, 9, 13, 19, 24, 
27, 29, 33, 36, 39, 42, 44, 46, 47, 49, 51 , 52, 54, 59, 61 , 63 and 65 are independent 
claims; claims 57, 58, 66 and 67 are cancelled; and, claims 1 , 5, 9, 27, 29, 31 , 33, 36, 
39, 46, 47, 49, 54 and 65 are newly amended. This action is made Final. 

3. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Claim Rejections - 35 USC §112 

4. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

5. Claim 39 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. The phrase "each of said segment is identification as likely 
including a play of said baseball" of lines 3-4 of page 12 renders the claim indefinite. 
The examiner will interpret the phrase to mean: each of the segment is identified as 
likely including a play of baseball. 

The claims are generally narrative and indefinite, failing to conform with current 
U.S. practice. They appear to be a literal translation into English from a foreign 
document and are replete with grammatical and idiomatic errors. 
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Claim Rejections - 35 USC § 102 

6. Claims 33, 42 and 43 are rejected under 35 U.S.C. 102(b) as being anticipated 
by "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima"). 

As per claim 33, Kawashima teaches a method of processing a video including 
baseball comprising: 

identifying a plurality of segments of the baseball video, wherein the start of the 
segment is first identified for at least one of the segments (pp. 871-873, sections 1.1, 
1 .2, 2.1 and 2.2; the start of the segment is identified by a start time based upon when 
the ball is put into play) then wherein said identifying for the end of at least one of said 
segments is based upon detecting a plurality of scene changes, where the first scene 
change detected after the start of said at least one of said segments is free from being 
selected as said end of said at least one of said segments, where each of the segments 
includes a plurality of frames of the video (pp. 871-873, sections 1.1-2.2; wherein the 
scene change is based upon calculating the value from the count of pixels whose 
intensity change in successive frames are larger than a threshold): and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1.2; i.e. the indexed video segments is a digest of the game or summary of 
the video, a./c.a. compressed play). 

As per claim 42, Kawashima teaches a method of processing a video including 
baseball comprising: 
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(a) identifying a plurality of segments of the video, wherein the identifying for at 
least one of the segments includes detecting the start of the segment (pp. 871-873, 
sections 1.1, 1.2, 2.1 and 2.2) based, at least in part, upon processing of a frame of the 
video, where each of the segments includes a plurality of frames of the video wherein 
the processing characterizes whether a batter is sufficiently close to at least one of a 
catcher and an umpire proximate home base (pg. 871, lines 25-27); and 

creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

As per claim 43, Kawashima teaches a method of processing a video including 
baseball wherein at least one of the regions is generally brown and at least one of the 
regions is generally green are inherent in a baseball video given that the layout of a 
baseball field generally consists of the colors brown and green (Abstract; pp. 872, 
section 2.1.3). 

7. Claims 47, 48, 59 and 60-64 are rejected under 35 U.S.C. 102(e) as being 
anticipated by "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. 
("Rui"). 

As per claim 47, Rui teaches a method of processing a video including baseball 
comprising identifying a plurality of segment of the video wherein each of the segments 
includes a play of baseball, creating a summarization of the video based upon analysis 
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of the video by including the plurality of segments wherein the summarization includes 
fewer frames than the video (Abstract) and removing at least one of the segments from 
the summary based, at least in part, upon audio information related to the at least one 
of the segments (pg. 105, right column, lines 24-33). 

As per claim 48, Rui teaches a method of processing a video including baseball 
wherein the audio information is obtained exclusively from a temporal analysis 
(Abstract; pg. 105, right column, lines 24-33; 3 rd paragraph, pg. 107 through 3.1.5 
Summary, pg. 108; using audio-track features, highlights of exciting portions of a 
baseball video is obtained, so that users can skip the boring parts thereby reducing the 
viewing time). 

As per claim 59, Rui teaches a method of processing a video comprising a 
service provider identifying: a summary of a video and a description of the summary 
wherein the summary and the description is provided to a remote user in response to 
receiving a request from the remote user (pg. 105, "Introduction"; a video summary is 
transmitted in response to user's request via indices/descriptions). 

As per claim 60, Rui teaches a method of processing a video wherein the user 
request the summary with a browser (pg. 105, left column, line 26 through right column, 
line 2). 

Claims 61 and 63 are individually similar in scope to claim 59 and are therefore 
rejected under similar rationale. 

Claims 62 and 64 are individually similar in scope to claim 60 and are therefore 
rejected under similar rationale. 
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Claim Rejections - 35 USC §103 

8. Claims 1-5, 7-12, 29 and 31-32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Indexing of Baseball Telecast for Content-based Video Retrieval", 
Kawashima et al. ("Kawashima") in view of Okayama et al. ("Okayama"). 

As per claim 1, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video based upon an event, wherein 
the event is characterized by a start time based upon when the ball is put into play and 
an end time based upon when the ball is considered out of play, where each of the 
segments includes a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 
and 2.2; e.g. the at bat event comprising of a start point in time slightly before the 
pitching and end point in time slightly after the catcher catches the ball if the ball is 
struck out and after the ball is thrown to a baseman if the ball is hit): and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1.2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

However, Kawashima does not explicitly disclose the event being characterized 
in a manner free from using a model sequence of frames to determine the approximate 
frame that the ball is put into play. Okayama teaches a method of processing a video 
wherein an event is characterized in a manner free from using a model sequence of 
frames to determine the approximate frame that the ball is put into play (fig. 4; section 
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[0096]). Therefore, it would have been obvious to an artisan at the time of the invention 
to include Okayama's teaching of an event being characterized in a manner free from 
using a model sequence of frames to determine the approximate frame that a ball is put 
into play in a method of processing a video to Kawashima's teaching of an event being 
characterized by a start time based upon when a ball is put into play and an end time 
based upon when it is considered out of play in a method of processing a video in order 
to provide users with an information extracting device for extracting information that is 
not restrictive to baseball only and that encompasses a wider range of video content. 

As per claim 2, the modified Kawashima teaches a method of processing a video 
including baseball wherein the event is defined by the rules of baseball (Kawashima: pp. 
871-873, sections 1 .1-2.1 .4; events such as scenes in which a batter was struck out or 
got a hit or a home run is defined by the rules of baseball using a spotting technique 
comprising a search of the minimal warp function by comparing input video sequence 
with pitching/batting model sequences). 

As per claim 3, the modified Kawashima teaches a method of processing a video 
including baseball wherein the start time is temporally proximate a baseball pitch 
(Kawashima: pg. 872, lines 10-11). 

As per claim 4, the modified Kawashima teaches a method of processing a video 
including baseball wherein the end time is temporally proximate to the batter missing 
the ball with a bat (Kawashima: pg. 872, lines 12-15). 

As per claim 5, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segment of the video, where each of the 
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segments includes a plurality of frames of the video, based upon a series of activities 
defined by the rules of baseball (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; series of 
activities such as scenes in which a batter was struck out or got a hit or a home run is 
defined by the rules of baseball using a spotting technique comprising a search of the 
minimal warp function by comparing input video sequence with pitching/batting model 
sequences) that could potentially result in at least one of a score, preventing a score, 
advancing a team toward a score, preventing advancing a team toward a score or 
creating a summarization of the video by including the plurality of segments where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. 
the indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play). 

However, Kawashima does not explicitly disclose the segments being free from 
including a batter swinging a bat. Okayama teaches a method of processing a video 
wherein segments being free from including a batter swinging a bat (fig. 4; section 
[0096]). Therefore, it would have been obvious to an artisan at the time of the invention 
to include Okayama's teaching of segments being free from including a batter swinging 
a bat in a method of processing a video to Kawashima's teaching of identifying a 
plurality of segments of a video based upon a series of activities defined by the rules of 
baseball in a method of processing a video in order to provide users with an information 
extracting device for extracting information that is not restrictive to baseball only and 
that encompasses a wider range of video content. 
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As per claim 7, the modified Kawashima teaches a method of processing a video 
including baseball wherein the activities are determined based upon the color 
characteristics of the video (Kawashima: pp. 872-873, section 2.1.3; activities are 
spotted by calculating the value from the count of pixels whose intensity change in 
successive frames are larger than a threshold wherein pixels are painted/colored to 
form an image produced on the screen). 

As per claim 8, the modified Kawashima teaches a method of processing a video 
including baseball wherein the activities are determined based upon scene changes 
(Kawashima: pp. 872-873; section 1.1-2.1.4; wherein an activity such as an at bat 
activity is a period from a basic scene to the next basic scene). 

As per claim 9, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video based upon detecting a play of 
the baseball game, wherein the identifying includes detecting the start of the play and 
detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences): and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
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section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a./c.a. compressed play). 

However, Kawashima does not explicitly disclose the event being characterized 
in a manner free from using a model sequence of frames to determine the approximate 
frame that the ball is put into play. Okayama teaches a method of processing a video 
wherein an event is characterized in a manner free from using a model sequence of 
frames to determine the approximate frame that the ball is put into play (fig. 4; section 
[0096]). Therefore, it would have been obvious to an artisan at the time of the invention 
to include Okayama's teaching of an event being characterized in a manner free from 
using a model sequence of frames to determine the approximate frame that a ball is put 
into play in a method of processing a video to Kawashima's teaching of an event being 
characterized by a start time based upon when a ball is put into play and an end time 
based upon when it is considered out of play in a method of processing a video in order 
to provide users with an information extracting device for extracting information that is 
not restrictive to baseball only and that encompasses a wider range of video content. 

As per claim 10, the modified Kawashima teaches a method of processing a 
video including baseball wherein the detecting the end of the play is based upon 
detecting the start of the play (Kawashima: pp. 872-873; section 1.1-2.1.4; wherein a 
play such as an at bat activity is a period from n end of a basic scene to the start of the 
next basic scene). 
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As per claim 1 1 , the modified Kawashima teaches a method of processing a 
video including baseball wherein the summarization identifies the plurality of segments 
of the video (Kawashima: pg. 872, section 1.2). 

As per claim 12, the modified Kawashima teaches a method of processing a 
video including baseball wherein the summarization is a summarized video comprising 
the plurality of segments excluding at least a portion of the video other than the plurality 
of segments (Kawashima: pg. 872, section 1.2). 

As per claim 29, Kawashima teaches a method of processing a video including 
baseball comprising: 

identifying a plurality of segments of the baseball video, wherein the identifying 
for the end of at least one of the segments is based upon detecting a scene change, 
where each of the segments includes a plurality of frames of the video (pp. 871-873, 
sections 1 .1 , 1 .2, 2.1 and 2.2; wherein an activity such as an at bat activity is a period 
from a basic scene to the next basic scene)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1 .2; i.e. the indexed video segments is a digest of the game or summary of 
the video, a.k.a. compressed play). 

Although Kawashima teaches identifying a plurality of segments of the baseball 
video comprising identifying the start of at least one of the segments using a model 
sequence of frames to determine the approximate frame that the ball is put into play 
(pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2), Kawashima does not explicitly disclose 
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identifying for the start of at least one of the segments being free from using a model 
sequence of frames to determine the approximate frame that the ball is put into play. 
Okayama teaches a method of processing a video comprising identifying for the start of 
at least one of the segments being free from using a model sequence of frames to 
determine the approximate frame that the ball is put into play (fig. 4; section [0096]). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Okayama's teaching of identifying for the start of at least one of the segments 
being free from using a model sequence of frames to determine the approximate frame 
that the ball is put into play in a method of processing a video to Kawashima's teaching 
of identifying the start of at least one of the segments using a model sequence of 
frames to determine the approximate frame that the ball is put into play in a method of 
processing a video in order to provide users with an additional implementation 
preference in video segmenting for extracting content. 

As per claim 31, the modified Kawashima teaches a method of processing a 
video including baseball wherein the identifying for the start of at least one of the 
segments is based upon detecting regions of color (Kawashima: pp. 872, section 2.1.3; 
i.e. the identifying for the start of at least one of the segments is based upon detecting 
regions of changing pixels/color). 

As per claim 32, the modified Kawashima teaches a method of processing a 
video including baseball wherein at least one of the regions is generally brown and at 
least one of the regions is generally green are inherent in a baseball video given that 
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the layout of a baseball field generally consists of the colors brown and green 
(Kawashima: Abstract; pp. 872, section 2.1.3). 

9. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Indexing 
of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of Okayama et al. ("Okayama") as applied to claim 5, and further 
in view of "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. 
("Rui"). 

As per claim 6, although the modified Kawashima teaches a method of 
processing a video including baseball wherein the summarization of the plurality of 
segments comprises a plurality of segments within the video (Kawashima: pg. 872, 
section 1 .2; the indexed video segments of the summarization of the plurality of 
segments is stored as a digest of the game), the modified Kawashima does not 
explicitly disclose the summarization of the plurality of segments to be in the same 
temporal order as the plurality of segments within the video. Rui teaches a method of 
processing a video including baseball wherein the summarization of the plurality of 
segments is in the same temporal order as the plurality of segments within the video 
(Abstract; section 5.4; Introduction; a method of allowing users to watch just the 
highlights of the exciting portions instead of the whole game due to time constraints, i.e. 
highlights are extracted automatically so that viewing time can be reduced). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Rui's 
method of processing a video including baseball wherein the summarization of the 
plurality of segments is in the same temporal order as the plurality of segments within 
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the vide.o to the modified Kawashima's method of processing a video including baseball 
wherein the summarization of the plurality of segments comprises a plurality of 
segments within the video so that the time in which sequential plays in a game is being 
viewed is reduced. 

10. Claims 13-26, 35, 39, 44, 45 and 51-53 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over "Indexing of Baseball Telecast for Content-based Video 
Retrieval", Kawashima et al. ("Kawashima") in view of "Multimedia Content Analysis", 
Wang et al. ("Wang"). 

As per claim 13, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon detecting the start of the play and 
detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences) and wherein 
the activities are determined based upon the color characteristics of the video (pp. 872- 
873, section 2.1 .3; activities are spotted by calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen)] and 
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(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1.2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose one of the region to be of a generally green 
color and at least one region of a generally brown color. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
the start of the plurality of segments is identified based upon detecting at least one 
region of a color such as green and at least one region of another color such as brown 
(pp. 32-33, "Visual Descriptors"; each segment at any level of the hierarchy, a set of 
audio and visual event-type Ds and annotation DSs, are used to characterize the 
segment or region comprising descriptors such as dominant color, in which green and 
brown is inherently inclusive as dominant colors of baseball, or color histogram 
wherein several types of histograms can be specified including common color 
histogram, which includes the percentage of each quantized color among al[ pixels in a 
segment or region, or instead of specifying the entire color histogram, one can specify 
the first few coefficients of the Haar transform of the color histogram). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Wang's 
identifying a plurality of video segments based upon detecting at least one region of a 
color such as green and at least one region of another color such as brown to 
Kawashima's identifying a plurality of video segments based upon color characteristics 
of the video as an additional visual descriptor in characterizing the video segment as 
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part of a goal of video segmentation in automatically grouping shots into what are 
perceived as "scenes". 

As per claims 14-16, the modified Kawashima teaches a method of processing a 
video including baseball comprising detecting at least two regions of the generally green 
color and/or two regions of generally brown color (Kawashima: Abstract; alluding to 
baseball video segmentation] Wang: pg. 33, left column, lines 20-26). 

As per claims 17 and 18, the modified Kawashima teaches a method of 
processing a video including baseball comprising color histograms used in identifying a 
plurality of segments of the video wherein the green color and the brown color being 
generally vertically aligned or having sufficient horizontal spatial extent are inherent in a 
baseball video given that the layout of a baseball field comprises of vertical alignment of 
green and brown colors (Kawashima: Abstract; alluding to baseball video segmentation: 
Wang: pg. 33, lines 6-17; pg. 33, lines 27-46). 

As per claim 19, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon detecting the start of the play and 
detecting the end of the play, where each of the segments includes a plurality of frames 
of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the 
play in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences) and wherein 
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the activities are determined based upon the color characteristics of the video (pp. 872- 
873, section 2.1 .3; activities are spotted by calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen)] and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose at least three regions of horizontally oriented 
regions of colors. Wang teaches a method of processing a video comprising identifying 
a plurality of segments of the video, wherein videos are segmented based upon regions 
of horizontally oriented regions of colors (pp. 32-33, 'Visual Descriptors"; i.e. color 
histograms are used in identifying a plurality of segments of the video wherein regions 
of 3 horizontally oriented regions of colors are inherent in a baseball video given the 
layout of a baseball field) Therefore, it would have been obvious to an artisan at the 
time of the invention to include Wang's method of processing a video comprising 
identifying a plurality of segments of the video, wherein videos are segmented based 
upon regions of horizontally oriented regions of colors to Kawashima's method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
videos are segmented based upon the change in color characteristics of the video as an 
additional visual descriptor in characterizing the video segment as part of a goal of 



Application/Control Number: 09/934,004 Page 18 

Art Unit: 2174 

video segmentation in automatically grouping shots into what are perceived as 
"scenes". 

As per claims 20-22, the modified Kawashima teaches a method of processing a 
video including baseball wherein at least two of the regions have the same dominant 
color, at least three regions have different said colors and at least two regions are 
located on opposing sides of another said detected region of a different color 
(Kawashima: Abstract; alluding to baseball video segmentation] Wang: pg. 33, left 
column, lines 20-26; described are features that are all inherent to the layout of a 
baseball field), 

As per claim 23, the modified Kawashima teaches a method of processing a 
video including baseball wherein said one of said regions is generally brown and 
another of the regions is generally green are inherent in a baseball video given that the 
layout of a baseball field generally consists of the colors brown and green (Kawashima: 
Abstract; alluding to baseball video segmentation] Wang: pg. 33, left column, lines 20- 
26). 

As per claim 24, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play, where each of the segments includes a plurality of 
frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start 
of the play in which a batter was struck out or got a hit or a home run is defined by the 
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rules of baseball using a spotting technique comprising a search of the minimal warp 
function by comparing input video sequence with pitching/batting model sequences): 
and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1.2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon identifying at least three regions of sufficient spatial coherence and 
sufficient horizontal extent. Wang teaches a method of processing a video comprising 
identifying a plurality of segments of the video, wherein videos are segmented based 
upon shape/spatial geometry such as regions of sufficient spatial coherence and 
horizontal extent (pg. 33, lines 27-46) Therefore, it would have been obvious to an 
artisan at the time of the invention to include Wang's method of processing a video 
comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon shape/spatial geometry such as regions of sufficient spatial 
coherence and horizontal extent to Kawashima's method of processing a video 
including baseball comprising identifying a plurality of segments of the video, wherein 
the start of the plurality of segments is identified based upon identifying the start of the 
play and identifying the end of the play in order to provide users with an additional visual 
descriptor in characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 
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As per claim 25, the modified Kawashima teaches a method of processing a 
video including baseball wherein each said at least three regions is at least one of 
substantially generally green and substantially generally brown are inherent in a 
baseball video given that the layout of a baseball field generally consists of at least 
three regions of substantially generally green and substantially generally brown 
(Kawashima: Abstract; alluding to baseball video segmentation] Wang: pg. 33, left 
column, lines 20-26). 

As per claim 26, the modified Kawashima teaches a method of processing a 
video including baseball wherein the three regions being generally vertically spaced 
apart are inherent in a baseball video given that the layout of a baseball field comprises 
of three regions of generally vertical spacing (Kawashima: Abstract; alluding to baseball 
video segmentation] Wang: pg. 33, lines 27-46). 

As per claim 35, although Kawashima teaches a method of processing a video 
including baseball comprising identifying a plurality of segments of the baseball video, 
wherein said identifying for the end of at least one of said segments is based upon 
detecting a plurality of scene changes, where each of the segments includes a plurality 
of frames of the video (pp. 871-873, sections 1.1-2.2; wherein the scene change is 
based upon calculating the value from the count of pixels whose intensity change in 
successive frames are larger than a threshold), Kawashima does not explicitly disclose 
one of the region to be of a generally green color and at least one region of a generally 
brown color. Wang teaches a method of processing a video comprising identifying a 
plurality of segments of the video, wherein the start of the plurality of segments is 
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identified based upon detecting at least one region of a color such as green and at least 
one region of another color such as brown (pp. 32-33, "Visual Descriptors"; each 
segment at any level of the hierarchy, a set of audio and visual event-type Ds and 
annotation DSs, are used to characterize the segment or region comprising descriptors 
such as dominant color, in which green and brown is inherently inclusive as dominant 
colors of baseball, or color histogram wherein several types of histograms can be 
specified including common color histogram, which includes the percentage of each 
quantized color among al[ pixels in a segment or region, or instead of specifying the 
entire color histogram, one can specify the first few coefficients of the Haar transform of 
the color histogram). Therefore, it would have been obvious to an artisan at the time of 
the invention to include Wang's identifying a plurality of video segments based upon 
detecting at least one region of a color such as green and at least one region of another 
color such as brown to Kawashima's identifying a plurality of video segments based 
upon color characteristics of the video as an additional visual descriptor in 
characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

As per claims 39 and 44, Kawashima teaches a method of processing a video 
including baseball comprising identifying a plurality of segments of the video wherein 
each of the segments is identified as likely including a play of baseball (pp. 871-873, 
sections 1.1, 1 .2, 2.1 and 2.2; e.g. a play of baseball/the at bat event is characterized by 
a start point in time slightly before the pitching and end point in time slightly after the 
catcher catches the ball if the ball is struck out and after the ball is thrown to a baseman 



Application/Control Number: 09/934,004 Page 22 

Art Unit: 2174 

if the ball is hit) and wherein each of the segments includes detecting a play of the 
baseball game (Abstract) and creating a summarization of the video by including the 
plurality of segments, where the summarization includes fewer frames than the video 
(Abstract; pg. 872, section 1 .2; i.e. the indexed video segments is a digest of the game 
or summary of the video, a.k.a. compressed play). Kawashima does not explicitly 
disclose detecting a commercial or a segment that has a temporally sufficiently short 
duration within the video and creating a summarization of the video wherein the plurality 
of segments of the video sufficiently temporally close to the commercial are free from 
being included within the summarization, where the segment that has a temporally 
sufficiently short duration, or separating/removing the identified segment from a 
summarization. Wang teaches a method of processing a video comprising identifying a 
plurality of segments of the video, wherein videos are segmented based upon detecting 
a commercial within the video and creating a summarization of the video wherein the 
plurality of segments of the video sufficiently temporally close to the commercial are free 
from being included within the summarization, where the segment that has a temporally 
sufficiently short duration, or separating/removing the identified segment from a 
summarization (pg. 21, right column; pg. 29, right column; separation of interested video 
portions and commercials). Therefore, it would have been obvious to an artisan at the 
time of the invention to include Wang's method of detecting a commercial within the 
video and creating a summarization of the video wherein the plurality of segments of the 
video sufficiently temporally close to the commercial are free from being included within 
the summarization, where the segment that has a temporally sufficiently short duration, 
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or separating/removing the identified segment from a summarization to Kawashima's 
method of detecting a play of the baseball game in order to provide users with additional 
criteria in content-based video retrieval. 

As per claim 45, although the modified Kawashima teaches a method of 
processing a video including baseball wherein the commercial is detected based upon a 
change in color and calculation of the most dominant color (Wang: pg. 21 , left column, 
lines 1-11; pg. 26, line 33 through right column, line 10; commercials are detected 
based upon changes in color, wherein color includes black), the modified Kawashima 
does not explicitly disclose the commercial being detected based upon at least one 
substantially black frame. Official Notice is taken that detection of a commercial frame 
adjacent to a black frame is well known in the art. Therefore, it would have been 
obvious to an artisan at the time of the invention to include detection of a commercial 
frame adjacent to a black frame to the modified Kawashima's method of detection of a 
commercial based upon a change in color and calculation of the most dominant color in 
order to allow for errors in video editing where a black frame is left uncut. 

As per claim 51 , Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play, where each of the segments includes a plurality of 
frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start 
of the play in which a batter was struck out or got a hit or a home run is defined by the 
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rules of baseball using a spotting technique comprising a search of the minimal warp 
function by comparing input video sequence with pitching/batting model sequences): 
and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1.2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon detecting different textures. Wang teaches a method of processing a video 
comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon detecting different textures (pg. 33, "Texture" section) 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Wang's method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon detecting different 
textures to Kawashima's method of processing a video including baseball comprising 
identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the 
end of the play in order to provide users with an additional visual descriptor in 
characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

As per claim 52, Kawashima teaches a method of processing a video including 
baseball comprising: 
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(a) identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play, where each of the segments includes a plurality of 
frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start 
of the play in which a batter was struck out or got a hit or a home run is defined by the 
rules of baseball using a spotting technique comprising a search of the minimal warp 
function by comparing input video sequence with pitching/batting model sequences)] 
and 

(b) creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified 
based upon the video being free from substantial translational motion. Wang teaches a 
method of processing a video comprising identifying a plurality of segments of the video, 
wherein videos are segmented based upon the video being free from substantial 
translational motion (pg. 22, left column, 2 nd paragraph). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Wang's method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
videos are segmented based upon the video being free from substantial translational 
motion to Kawashima's method of processing a video including baseball comprising 
identifying a plurality of segments of the video, wherein the start of the plurality of 
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segments is identified based upon identifying the start of the play and identifying the 
end of the play in order to provide users with an additional visual descriptor in 
characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 

As per claim 53, Kawashima teaches a method of processing a video including 
baseball comprising: 

(a) identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a scene 
change, where each of said segments includes a plurality of frames of the video (pp. 
871-873, sections 1.1, 1 .2, 2.1 and 2.2; wherein an activity such as an at bat activity is a 
period from a basic scene to the next basic scene): and 

(b) creating a summarization of the video by including said plurality of segments, 
where the summarization includes fewer frames than said baseball video (Abstract; pg. 
872, section 1.2; i.e. the indexed video segments is a digest of the game or summary of 
the video, a.k.a. compressed play). 

Kawashima does not explicitly disclose not using the scene change to identify the end 
of the at least on of said segments if the temporally relevant portion of the at least one 
said segment includes substantially horizontal motion. Wang teaches not using the 
scene change to identify the end of the at least on of said segments if the temporally 
relevant portion of the at least one said segment includes substantially horizontal motion 
(pg. 21, right column, lines 1-2; pg. 22, left column, lines 3-35; pg. 23, right column, 
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lines 18-22, lines 34-37 and lines 53-55; pg. 28, right column, lines 33-34; pg. 33, lines 
13-48). Therefore, it would have been obvious to an artisan at the time of the invention 
to include Wang's method of not using the scene change to identify the end of the at 
least on of said segments if the temporally relevant portion of the at least one said 
segment includes substantially horizontal motion to Kawashima's method identifying a 
plurality of segments of the baseball video, wherein said identifying for the end of at 
least one of said segments is based upon detecting a scene change so that a desired 
scene change is not missed. 

11. Claims 27 and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima") in view of "Multimedia Content Analysis", Wang et al. ("Wang"), and 
further in view of Okayama et al. ("Okayama"). 

As per claim 27, Kawashima teaches a method of processing a video including 
baseball comprising: 

identifying a plurality of segments of the video based upon an event, wherein the 
identifying for at least one of the segments includes detecting the start of the segment 
based upon processing of a first single frame of the video, where each of the segments 
includes a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2); 
and 

creating a summarization of the video by including the plurality of segments, 
where the summarization includes fewer frames than the video (Abstract; pg. 872, 
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section 1 .2; i.e. the indexed video segments is a digest of the game or summary of the 
video, a.k.a. compressed play). 

Kawashima does not explicitly disclose verifying that said first single frame is an 
appropriate start of the segment based upon processing of another single frame 
temporally relevant to the first single frame. Wang teaches verifying that said first single 
frame is an appropriate start of the segment based upon processing of another single 
frame temporally relevant to the first single frame (pg. 22, left column, lines 1-8). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Wang's verifying that said first single frame is an appropriate start of the 
segment based upon processing of another single frame temporally relevant to the first 
single frame to Kawashima's start of the segment based upon processing of another 
single frame temporally relevant to the first single frame in order to reduce errors in 
segmenting related scenes. 

However, Kawashima and Wang still do not explicitly disclose detecting a 
segment based upon processing of a first single frame of the video independently of 
other frames. Okayama teaches a method of processing a video comprising detecting a 
segment based upon processing of a first single frame of the video independently of 
other frames (fig. 4; section [0096]). Therefore, it would have been obvious to an artisan 
at the time of the invention to include Okayama's teaching of detecting a segment 
based upon processing of a first single frame of the video independently of other frames 
in a method of processing a video to Kawashima's teaching of detecting a segment 
based upon processing of a first single frame of the video in a method of processing a 
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video in order to provide users with an additional implementation preference in video 
segmenting for extracting content. 

As per claim 28, the modified Kawashima teaches a method of processing a 
video including baseball wherein in said identifying based upon detecting at least one 
region of generally brown and at least one region of generally green are inherent in a 
baseball video given that the layout of a baseball field generally consists of the colors 
brown and green (Kawashima: Abstract; alluding to baseball video segmentation: 
Warfg: pg. 33, left column, lines 20-26). 

12. Claim 30 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of "Multimedia Content Analysis", Wang et al. ("Wang"). 

As per claim 30, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the baseball video, wherein 
the scene change is based upon calculating the value from the count of pixels whose 
intensity change in successive frames are larger than a threshold (pp. 872-873, section 
2.1 .3). Kawashima does not teach identifying the scene change based upon a 
histogram. Wang teaches a method of processing a video wherein identifying the scene 
change in video segmenting is based upon a histogram (Wang: pg. 33, left column, 
lines 6-17; pg. 33, right column lines 3-12). Therefore, it would have been obvious to an 
artisan at the time of the invention to include Wang's method of processing a video 
wherein scene change in video segmenting is based upon a histogram to Kawashima's 
method of processing a video wherein scene change is based upon calculating the 
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value from the count of pixels whose intensity change in successive frames are larger 
than a threshold in order to provide users with an implementation preference. 
13. Claim 34 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of Okayama et al. ("Okayama") as applied to claim 29, and 
further in view of 'Video summarization using Hidden Markov Model", Huang et al. 
("Huang"). 

As per claim 34, although the modified Kawashima teaches a method of 
processing a video including baseball comprising identifying a plurality of segments of 
the baseball video, wherein said identifying for the end of at least one of said segments 
is based upon detecting a plurality of scene changes, where each of the segments 
includes a plurality of frames of the video (Kawashima: pp. 871-873, sections 1.1-2.2; 
wherein the scene change is based upon calculating the value from the count of pixels 
whose intensity change in successive frames are larger than a threshold), the modified 
Kawashima does not explicitly disclose the frame is identified as primarily the field at the 
first scene change. Huang teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the baseball video, wherein the frame is 
identified as primarily the field at the first scene change (pg. 473, "Introduction"; e.g. 
distance view of batter shot, defense of the outfield). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Huang's method of 
processing a video including baseball comprising identifying a plurality of segments of 
the baseball video, wherein the frame is identified as primarily the field at the first scene 
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change to the modified Kawashima's method of processing a video including baseball 
comprising identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a 
plurality of scene changes, where each of the segments includes a plurality of frames of 
the video in order to provide users with additional criteria in content-based video 
retrieval. 

14. Claims 36-38 and 40-41 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Indexing of Baseball Telecast for Content-based Video Retrieval", 
Kawashima et al. ("Kawashima") in view of "Multimedia Content Analysis", Wang et al. 
("Wang") and further in view of "Automatically Extracting Highlights for TV Baseball 
Programs", Rui et al. ("Rui"). 

As per claim 36, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video wherein each of the 
segments is identified as a play (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; the 
segment is identified by a start time based upon when the ball is put into play and an 
end time based upon when the ball is considered out of play, e.g. the at bat event 
comprising of a start point in time slightly before the pitching and end point in time 
slightly after the catcher catches the ball if the ball is struck out and after the ball is 
thrown to a baseman if the ball is hit) and wherein each of the segments includes a 
plurality of frames of the video (Abstract) and creating a summarization of the video by 
including the plurality of segments, where the summarization includes fewer frames 
than the video (Abstract; pg. 872, section 1 .2; i.e. the indexed video segments is a 
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digest of the game or summary of the video, a.k.a. compressed play). Kawashima does 
not explicitly disclose identifying a plurality of segments that are temporally separated 
by a sufficiently short duration and then connecting the identified plurality of segments. 
Wang teaches a method of processing a video comprising identifying a plurality of 
segments that are temporally separated by a sufficiently short duration and then 
connecting the identified plurality of segments (pg. 21, right column; pg. 29, right 
column; separation of interested video portions and commercials). Therefore, it would 
have been obvious to an artisan at the time of the invention to include Wang's method 
of identifying a plurality of segments that are temporally separated by a sufficiently short 
duration to Kawashima's method of detecting a play of the baseball game in order to 
provide users with additional criteria in content-based video retrieval. 

However, Kawashima and Wang still do not explicitly disclose connecting the 
identified plurality of segments. Rui teaches a method of processing a video including 
baseball comprising of connecting the identified plurality of segments so that the 
summary is in the same temporal order as the plurality of segments within the video 
(Abstract; section 5.4; Introduction; a method of allowing users to watch just the 
highlights of the exciting portions instead of the whole game due to time constraints, i.e. 
highlights are extracted automatically so that viewing time can be reduced). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Rui's 
method of processing a video including baseball comprising of connecting the identified 
plurality of segments to the Kawashima and Wang's method of processing a video 
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including baseball comprising of a plurality of segments within the video so that the time 
in which sequential plays in a game is being viewed is reduced. 

As per claims 37-38 and 40-41, the modified Kawashima teaches a method of 
processing a video including baseball wherein the connecting includes discarding the 
frames of the video between the identified plurality of segments and wherein the 
connecting results in a single segment that includes the identified plurality of segments 
together with the frames of the video between the identified plurality of segments 
(Wang: pg. 21, right column; pg. 29, right column; separation of interested video 
portions and commercials', Rui: Abstract; section 5.4; Introduction; a method of allowing 
users to watch just the highlights of the exciting portions instead of the whole game due 
to time constraints, i.e. highlights are extracted automatically so that viewing time can 
be reduced). 

15. Claim 46 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of "Detection of Slow-Motion Replay Segments in Sports Video 
for Highlights Generation", Pan et al. ("Pan"), and further in view of Okayama et al. 
("Okayama"). 

As per claim 46, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video wherein each of the 
segments includes a play of baseball wherein the segments include full-speed plays 
and creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video, where a user may select from 
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the summarization including only full-speed plays (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play where users may select a full-speed play segment among the plurality 
of segments). Kawashima does not disclose segments that include slow motion plays of 
the full-speed plays and creating a summarization where a user may select from the 
summarization comprising only of slow motions plays. Pan teaches a method of 
processing a video including baseball comprising identifying a plurality of segments of 
the video wherein each of the segments includes a play of baseball ("Introduction", left 
column) wherein the segments include slow motion plays of the full-speed plays 
("Introduction", right column; in processing the video, slow motion plays of the full-speed 
plays and full-speed plays are identified) and users may select from the summarization 
comprising only of slow motions plays. Therefore, it would have been obvious to an 
artisan at the time of the invention to include Pan's segments that include slow motion 
plays of the full-speed plays and creating a summarization where a user may select 
from the summarization comprising only of slow motions plays to Kawashima's 
segments that include full-speed plays and creating a summarization where a user may 
select from the summarization comprising only of full-speed plays in order to provide 
user with the ability to capture inherently important events. 

However, Kawashima and Pan still do not explicitly disclose identifying a 
segment in a manner free from using a model sequence of frames to determine the 
approximate frame that the play starts. Okayama teaches a method of processing a 
video comprising identifying a segment in a manner free from using a model sequence 
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of frames to determine the approximate frame that the play starts (fig. 4; section [0096]). 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Okayama's teaching of identifying a segment in a manner free from using a 
model sequence of frames to determine the approximate frame that the play starts in a 
method of processing a video to Kawashima and Pan's teaching of detecting a segment 
based upon processing of a first single frame of the video in a method of processing a 
video in order to provide users with an additional implementation preference in video 
segmenting for extracting content. 

16. Claims 49 and 50 are rejected under 35 U.S.C. 102(e) as being anticipated by 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in view 
of "Multimedia Content Analysis", Wang et al. ("Wang"). 

As per claim 49, although Rui teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video, where the 
identifying is based upon analysis of frames of the video, wherein each of the segments 
includes a play of baseball, creating a summarization of the video by including the 
plurality of segments wherein the summarization includes fewer frames than the video 
and the duration of at least one of the segments from the summary is based, at least in 
part, upon audio information related to the at least on of the segments (Abstract; pg. 
105, right column, lines 24-33), Rui does not explicitly disclose modifying the duration of 
at least one of the segments from the summary based, at least in part, upon audio 
information related to the at least on of the segments. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, creating a 
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summarization of the video by including the plurality of segments wherein the 
summarization includes fewer frames than the video and modifying the duration of at 
least one of the segments from the summary based, at least in part, upon audio 
information related to the at least on of the segments (pg. 29, left column, lines 49-53; 
pg. 30, left column, lines 6-22). Therefore, it would have been obvious to an artisan at 
the time of the invention to include Wang's modifying the duration of at least one of the 
segments from the summary based, at least in part, upon audio information related to 
the at least on of the segments to the method of Rui wherein the duration of at least one 
of the segments from the summary based, at least in part, upon audio information 
related to the at least on of the segments in order to provide users with a more 
customized method of processing a video. 

As per claim 50, the modified Rui teaches a method of processing a video 
including baseball wherein the audio information is obtained exclusively from a temporal 
analysis (Rui: Abstract; pg. 105, right column, lines 24-33; 3 rd paragraph, pg. 107 
through 3.1 .5 Summary, pg. 108; using audio-track features, highlights of exciting 
portions of a baseball video is obtained, so that users can skip the boring parts thereby 
reducing the viewing time). 

17. Claim 54 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in view 
of Kim et al. ("Kim"). 

As per claim 54, although Rui teaches a method of processing a video 
comprising providing a description of the video, identifying a plurality of segments of the 
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video based upon the description (pg. 105, "Introduction"; provided are 
indices/descriptions of the video) and generating another video based upon the 
identified segments, wherein the another video has less frames than the video (pg. 105, 
"Introduction"; a second video comprises of highlights of a first video wherein the 
second video consists of less frames than the first and is based upon the exciting 
portions/identified segments), Rui does not explicitly disclose defining a segment of a 
video using TV-Anytime compliant description scheme. Kim teaches a method of 
processing a video comprising defining a segment of a video using TV-Anytime 
compliant description scheme. Therefore, it would have been obvious to an artisan at 
the time of the invention to include Kim's teaching of defining a segment of a video 
using TV-Anytime compliant description scheme in a method of processing a video to 
Rui's teaching of identifying segments of a video based upon a description in a method 
of processing a video as an implementation preference. 

18. Claims 55 and 56 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in 
view of Kim et al. ("Kim"). 

As per claims 55 and 56, although the modified Rui teaches a method of 
processing a video wherein the video is in accordance with the channel capacity of a 
transmission medium and of a storage device (Rui: pg. 105, left column, lines 26-35), 
the modified Rui does not explicitly disclose the video is transcoded in accordance with 
the channel capacity of a transmission medium and transcoded in accordance with the 
channel capacity of a storage device. Official Notice is taken that transcoding video in 
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accordance with the channel capacity of a transmission medium and transcoding in 
accordance with the channel capacity of a storage device is well known in the art. 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include a method of processing a video wherein the video is tanscoded in accordance 
with the channel capacity of.a transmission medium and of a storage device to Rui's 
method of processing a video wherein the video is in accordance with the channel 
capacity of a transmission medium and of a storage device so that users may view the 
segments of video with the associated description in various formats. 
19. Claim 65 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of Kim et al. ("Kim"). 

As per claim 65, although Kawashima teaches a method of processing a video 
comprising generating a summary of the video comprising a plurality of segments of the 
video, where each of the segments includes a plurality of framesj(Abstract), providing a 
description of the video, associating a portion of the description with a first one of the 
segments and associating another portion of the description with a second one of the 

segments (pp. 871 and 873, sections 1.1 and 2.2; provided are Indices into scenes), 

• \ " 

Kawashima does not explicitly disclose defining a segment of a video using TV-Anytime 
compliant description scheme. Kim teaches a method of processing a video comprising 
defining a segment of a video using TV-Anytime compliant description scheme. 
Therefore, it would have been obvious to an artisan at the time of the invention to 
include Kim's teaching of defining a segment of a video using TV-Anytime compliant 
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descriptioixscheme in a method of processing a video to Kawashima's teaching of 
identifying segments of a video based upon a description in a method of processing a 
video as an implementation preference. 

Response to Arguments 

20. Applicant's arguments with respect to claims /I -45, 47-50, 52-58 and 60-65 have 
been considered but are moot in view of the new ground(s) of rejection, except for the 
following: \ 
Applicant argued: 

(a) Claim 13 patentably distinguishes over Kawashima and Wang by claiming 
identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting at least one region of a generally green 
color and at least one region of a generally brown color. . 

(b) Claim 19 patentably distinguishes over Kawashima and Wang by claiming 
identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting at least tee. regions of horizontally 

oriented regions of colors, where each of the segmenis includes a plurality of frames of 

{- 

the video. ; { 

(c) Claim 24 patentably distinguishes over Kawashima and Wang by claiming 

i 

identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting at least thee regions of sufficient spatial 
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coherence and horizontal: extent, where each of the segments includes a plurality of 
frames of the video. 

(d) Claim 33 patentably distinguishes over Kawashima by claiming identifying a 
plurality of segment of the baseball video, wherein the start of the s jqmen . is first 
identified for at least one of the segments . 

(e) Claim 36 patentably distinguishes over Kawashima and Wang by claiming 
identifying a plurality of segments of the video, where each of the segments is identified 
as a play . 

(f) Claim 39 patentably distinguishes over Kawashima and Wang by claiming 
identifying a plurality of segments of the video, where each of the segments i s identified 
as likely including a play of baseball . 

(g) Claim 42 patentably distinguishes over Kawashima by claiming wherein the 
processing characterizes whether a batter is sufficiently close to at least one of a 
catcher and an umpire proximate home base. 

(h) Claim 44 patentably distinguishes over Kawashima and Wang by claiming 
identifying a plurality of segments of the video, where each of the segments is identified 
as likely including a play of baseball, detecting a commercial within the video, ancJ 
creating a summarization of the video by including the plurality of segments, whefe the . 
summarization includes fewer frames than the video, where the plurality of segmen s of 
the video sufficiently temporally dose to the commercial are free from being included 
within the summarization. 
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(i) Claim 47 patentably distinguishes over RUI by claiming creating a * 
summarization of the video based upon analysis of the video . 

(j) Claim 61 patentably distinguishes over Rui by claiming a service provider 
storing a summary of a video that includes baseball and the service provider providing 
the summary in the form of streaming video to a user that is interconnected to the 
service provider by a network, in response to receiving a request from the user. 

(k) Claim 63 patentably distinguishes over Rui by claiming a service provider 
storing a summary of a video that includes baseball and the service provider providing 
the description to a user that is interconnected to the service provider by a network in 
response to receiving a request from the user. 

The examiner disagrees for the following reasons: 

Per (a), the Office Action clearly indicates that Kawashima teaches identifying a 
plurality of video segments based upon color characteristics of the video (pp. 871-873," 
sections 1.1, 1.2, 2.1 and 2.2; pp. 872-873, section 2.1.3), while the teaching extracted 
from Wang is for identifying a plurality of video segments based upon detecting at least 
one region of a color such as green and at least one region of another color such as 
brown (pp. 32-33). 

Per (b), Kawashima teaches a method of processing a video comprising 
identifying a plurality of segments of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 
2.2; pp. 872-873, section 2.1 .3). The teaching extracted from Wang is for a method of 
processing a video comprising identifying a plurality of segments of the video, wherein 
videos are segmented based upon regions of horizontally oriented regions of colors. 
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Per (c), Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and 
identifying the end of the play (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; Abstract; pg. 
872, section 1.2). The teaching extracted from Wang is for a method of processing a 
video comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon shape/spatial geometry such as regions of sufficient spatial 
coherence and horizontal extent (pg. 33, lines 27-46). 

Per (d), the modified Kawashima does teach that the start of the segment is first 
identified for at least one of the segments, i.e. the start of the segment is identified by a 
start time based upon when the ball is put into play (Kawashima: pp. 871-873, sections 
1.1, 1.2,2.1 and 2.2). 

Per (e), Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
includes detecting a play of the baseball game (Abstract), which also includes each of 
the segments being identified as a play (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. 
a play/bat event is characterized by a start point in time slightly before the pitching and 
end point in time slightly after the catcher catches the ball if the ball is struck out and 
after the ball is thrown to a baseman if the ball is hit). 

Per (f), Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
includes detecting a play of the baseball game (Abstract), which also includes each of 
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the segments being identified as likely including a play of baseball (pp. 871-873, 
sections 1.1, 1 .2, 2.1 and 2.2; e.g. a play of baseball/the at bat event is characterized by 
a start point in time slightly before the pitching and end point in time slightly after the 
catcher catches the ball if the ball is struck out and after the ball is thrown to a baseman 
if the ball is hit). 

Per (g), Kawashima teaches a method of processing a video for extracting video 
content using a model sequence of frames in comparing changes in a scene including 
extracting scenes in which a batter struck out wherein the processing characterized by 
whether a batter is sufficiently close to at least one of a catcher and an umpire 
proximate home base (pg. 871, lines 25-27) is inherent since the scene in which a 
batter struck out would include a batter being close to an umpire proximate home base 
to be in accordance with the rules of baseball. 

Per (h), Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
is identified as likely including a play of baseball (pp. 871-873, sections 1.1, 1.2, 2.1 and 
2.2; e.g. a play of baseball/the at bat event is characterized by a start point in time 
slightly before the pitching and end point in time slightly after the catcher catches the 
ball if the ball is struck out and after the ball is thrown to a baseman if the ball is hit) and 
creating a summarization of the video by including the plurality of segments, where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. 
the indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play). The teaching extracted from Wang is for a method of processing a 
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video comprising identifying a plurality of segments of the video, wherein videos are 
segmented based upon detecting a commercial within the video and creating a 
summarization of the video wherein the plurality of segments of the video sufficiently 
temporally close to the commercial are free from being included within the 
summarization, where the segment that has a temporally sufficiently short duration, or 
separating/removing the identified segment from a summarization (pg. 21, right column; 
pg. 29, right column; separation of interested video portions and commercials). 

Per (i), Rui inherently teaches analyzing the video since analysis is required to 
create a summarization of the video (pg. 105, right column, lines 24-33; described is a 
method of summarizing video content). 

Per (j), Rui does teach a service provider storing a summary of a video that 
includes baseball and the service provider providing the summary in the form of 
streaming video to a user that is interconnected to the service provider by a network, in 
response to receiving a request from the user (pg. 105, "Introduction"; a video summary 
is transmitted in response to user's request via indices/descriptions). 

Per (k), Rui does teach a service provider storing a summary of a video that 
includes baseball and the service provider providing the description to a user that is 
interconnected to the service provider by a network in response to receiving a request 
from the user (pg. 105, "Introduction"). 

Furthermore, the Office notes that applicant did not contest the factual assertion 
set forth under Official Notice in paragraph sixteen of section eleven and paragraph two 
of section sixteen of the Office Action of 5/21/04. 
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Conclusion 

21 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 
706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 
1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Inquires 

22. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Examiner Le Nguyen whose telephone number is (571) 
272-4068. The examiner can normally be reached on Monday - Friday from 7:00 am to 
3:30 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 
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The fax numbers for the organization where this application or proceeding is 
assigned are as follows: 

(703) 872-9306 [Official Communication] 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 



3900. 




LVN 

Patent Examiner 
March 30, 2005 
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